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The method that is the subject of this patent permits one to generate a new frame 
or several frames in a video sequence if information about the restored frame or frames is 
fully absent. In connection with the fact that the restorable frame must correspond to the 
video sequence scene, a specially developed motion estimation procedure, which takes 
into consideration the characteristics of the given video sequence, permits one to re- 
create the necessary frame: for example, the subject, the dynamic of the subject. The 
important feature of this method is the detection of the vectors that most precisely 
correspond to the movements that take place in the video sequence (true motion vectors). 
Moreover, the formation of intermediate frames takes place in such a way that there are 
no breaks in the frame caused by covering the blocks and displacing them beyond the 
boundaries of the frame. The method that is the subject of this patent also permits one to 
increase the frame rate due to the formation of intermediate frames, which permits one to 
attain greater continuity of movement in the video film. With the assistance of methods 
of extrapolation and search beyond the boundaries of the frame, a solution for the 
problem of the movement of the camera in the course of the development of the 
videofilm scene is obtained. 
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Claims: 



What is claimed is: 

1 . A method and device for the temporal interpolation of a video sequence. 

2. A system constituting an aggregate of procedures for the implementation of 
temporal interpolation of the frames of a video sequence and including: 

■ a device for the reading of two adjacent frames of a video sequence; 

■ a device for the detection and removal of frame defects (such as a dark band 
along the edge of the frame); 

■ a device for the implementation of a procedure for the broadening of a frame 
beyond the border for better interpolation of a new frame; 

■ a device for the splitting of the frames of a video into blocks for the further 
search for a displacement vector for each block; 

■ a device for the automatic construction of parameters for a system 
accommodating a specific video sequence during operation of the method 
that is the subject of this patent; 

■ a device for the implementation of a motion estimation procedure; 

■ a device for the formation and insertion of an interpolated frame or several 
interpolated frames; 

■ a device for the evaluation of the quality of the interpolation; 

■ a device for the selection of the type of interpolation; 

■ a device for the storage of data necessary for the operation of the method that 
is the subject of this patent; 

■ a device for the detection of scene changes in the video sequence; 

■ a device for the output of the prepared video sequence with increased fps. 

3. The method in claim 1 , including the locating of vectors that most precisely 
correspond to the movements taking place in the video sequence (true motion 
vectors). 

4. The method in claim 1, including the formation of intermediate frames in such a 
way that there are no breaks caused by the overlapping of blocks and their 
displacement outside the borders of the frame. 

5. The method in claim 1 , including a procedure for the resolution of the problem of 
"camera movement" with the help of extrapolation and the implementation of a 
search beyond the borders of the frame. 

6. The method in claim 1, an important aspect of which is the processing of the 
video sequence by two adjacent frames. 

7. The indicated method (from claim 1) permits one to insert an arbitrary number of 
frames between two adjacent frames. 

8. The given method (from claim 1) may be used for the processing of a restored 
video sequence after compression by some coding method. 

9. The presented method (from claim 1) and system (from claim 2) permit the 
processing of a source video sequence of any video format (SIF, QCIF, R601). 



10. The presented method (from claim 1) and system (from claim 2) permit the 
processing of the source video sequence into real time. 

1 1 . The method that is the subject of this patent (from claim 1) includes an evaluation 
of the quality of the interpolation, is based on the evaluation of the size of the 
displacement vectors for blocks with an error greater than some threshold. 

12. The presented method and system (claim 1, 2) automatically adjust to 
accommodate any format for the source video sequence. 

13. The presented method and system (claim 1, 2) do not require preliminary 
adjustment of parameters for a particular video sequence, due to the fact that 
during operation the given method automatically adjusts to accommodate the 
character of the motion in each source video sequence. 

14. The method in claim 1, which includes a procedure for the identification of scene 
change in the video film. 

15. Said method (from claim 1) and system (from claim 2) may be implemented both 
in the form of a software application and as hardware. 



Description 



Field of Invention: 

The given method may be applied to the realm of video compression, as well as to the 
realm of video processing. The method that is the subject of this patent may be applied 
in the realm of computer modeling of video sequences and video effects to those such as 
a change of the object's form or movement. For example, to decrease computer 
expenditures during the creation of a videofilm, if an object that moves or changes its 
form is present in the subject. 

SUMMARY OF THE INVENTION 

The given invention constitutes a method for the effective creation and insertion of 
supplementary frames into a video sequence. Upon the insertion of a frame the character 
of the movement of objects in the frame is taken into account. The given invention may 
be applied in a sufficiently broad circle of realms relating to the processing of video 
sequences. In the method that is the subject of this patent, with the exception of the 
procedure for frame insertion itself, procedures are implemented for the removal of dark 
bands along the edges of frames, a procedure for the extrapolation of a frame (that is, a 
search for a suitable block beyond the borders of the real frame), as well as a motion 
estimation procedure for the evaluation of movement in the new frame. 
The work of the given method may be briefly described thus: on the basis of two 
adjacent frames, an interpolated frame is filled in, with the use of methods and 
procedures described in the following sections. Prior to the filling in of the frame, the 



already existing adjacent frame are cleansed of dark bands along the edges and are 
broadened beyond the borders for better interpolation of the new frame. Further, with 
the help of the motion estimation procedure described below, we find the motion vectors, 
which are then used for the filling in of the new frame, using information from the two 
neighboring frames. After the creation of the new frame the border, if there was one, is 
re-established along the edge of the frame. 



BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 dramatically illustrates the process of inserting additional frames into the video 
sequence. 

In Figure 2 a block diagram of the operation of the temporal interpolation algorithm for a 
video sequence is presented. 

Figure 3 illustrates a example of the movement of an object in a video film. 

Figure 4a shows an example of a dark band along the edge of a frame. 

Figure 4b illustrates the process for filling in the dark band in a frame with pixels from 

the frame. 

Figure 5 illustrates the plan for frame extrapolation. 

Figure 6 illustrates the operation of the method that is the subject of this patent on an 
example of frames from the "Foreman" video sequence. 

Figure 7 illustrates the operation of the method that is the subject of this patent on an 

example of frames from the "Football" video sequence. 

Figure 8 illustrates in detail the broadening of blocks in a frame. 

Figure 9 shows the order of passing of the blocks. 

In Figure 10, the structure of a block search is described. 

DETAILED DESCRIPTION OF THE INVENTION 

The present invention is better understood upon consideration of following definitions. 
Definitions: 

Temporal interpolation: in the given invention is understood as a process of generation 
and insertion of one or several frames into a source video sequence. In the given 
invention, for better interpolation of the new generated frames an aggregate of specific 
procedures, such as the broadening of existing frames beyond their borders, the 
cleansing of frames of defects (for example, of the dark band along the edge of the 
frame), a motion estimation procedure and so forth, is called for. The procedures used 
are described in detail below. 

Interpolated frame: a frame, generated by the method that is the subject of this patent, 
for insertion between the already existing frames of a video sequence. 
Object motion: the change in location of an object over the length of the subject of a 
video sequence. 

Object deformation: the change in form of an object over the length of the subject of the 
video sequence. 

Motion vector: in the presented invention, the difference between the coordinates of the 
block in the preceding frame and the coordinates of the corresponding block found in the 
next frame. 



Motion estimation: the process of locating motion vectors for all blocks in the preceding 
frame. 

Adjacent frames: frames with the numbers N and N+l in a video sequence, as shown in 
figure 1 . 

Fps: the number of frames per second for a video sequence. The given amount may be 

different. The method that is the subject of this patent permits the increase in value of 

fps by a whole number of times for the source video sequence. 

Extrapolation: broadening of the frame beyond the borders by a specific number of 

pixels. 

The given algorithm is designed for the creation of an arbitrary number of frames 
between two adjacent frames of a video sequence. Depending on the dynamic of the 
subject of the video sequence, there may arise the necessity for the insertion of one 
frame or of several frames between two adjacent frames. The insertion of one or several 
frames helps to attain the smoothness of the movements of objects in the video film. 
Here it is possible to say several words about the different types of dynamic for 
development of the subject of the video sequence. There are video films with a different 
subject dynamic. In drawings 6 and 7 frames from the video sequences "Foreman" and 
"Football" are introduced; the given video sequences are films with a high dynamic of 
subject development, since in them the objects move very intensively, in this way for the 
duration of several frames a significant change of scene or subject takes place. With the 
help of the method that is the subject of this patent, several frames, shown in figure 6, 
were added between adjacent frames of the "Foreman" video sequence. Thus, between 
the original frames D and G the interpolated frames E and F were inserted, in order to 
make a transition to another scene with a landscape more smooth as well as in order to 
make more realistic the movement of the person who quickly nods and gesticulates. For 
the frames from the "Football" video sequence the situation is somewhat different: there 
is also rapid movement of objects, however in the given place the scene does not change, 
therefore one frame is added in order to make the movement of the objects (the falling of 
football players with a ball) more natural. 

Algorithm Description 

The given invention includes important aspects described below: 

1 . Reading of two adjacent frames. 

2. Identification and storage of the dark band along the edge of the frame. If the 
band is present in the frame (an example is shown in figure 4a), then the 
procedure for removing the band from the frame is applied (as shown in figure 
4b). 

3. Each frame is expanded beyond the borders by 8 pixels from each side of the 
frame as shown in figure 5, with the help of the extrapolation procedure described 
below. 

4. We split the next frame into quadratic blocks and for each block we shall find the 
corresponding block in the previous frame, with the help of the procedure 
described below. In this way, we obtain the displacement vector for each block. 



5. We decrease the length of the displacement vector by k/n times, where n is the 
coefficient for increasing fps, and 0<k<n is the number relative to the previous, 
inserted frame. 

6. Determination of the presence of a scene change with the help of the 
corresponding procedure. 

7. If there was no scene change, then the interpolated frame is filled with the help of 
the procedure for filling the interpolated frame on the basis of the calculated 
vectors of movement. 

If there was a scene change, then the pixels of the interpolating frame are filled by 
values in the way described in the procedure for making a decision as to the type 
of interpolation. 

8. We restore the dark band along the edges of the frame in the event that it was 
there initially. 

9. The obtained frame is inserted into the video sequence and is the output. 

Important aspects of this invention are further described in detail. 

Description of the procedure for the detection and removal of defects (dark bands) along 

the edges of the frame 

In some video sequences there is a dark band on the edges of the frame, which upon 
enlargement looks approximately the same as shown in figures 4a and 4b. 
As is obvious from the attached figures (4a and 4b), the dark band consists of bands 
which we perceive as black and bands, the brightness of which differs significantly from 
the brightness of the primary part of the frame, that is the much darker band. 
It is derived by experimentation that the average brightness value of the black band in 
the YUV color system does not exceed 20, and the brightness differential for the 
darkened band is not less than 35 (for video sequences on which testing has been 
conducted), that is, the difference between the average brightness of the dark band and 
the average brightness of the subsequent band of pixels, as a rule, is more than 35. 
In order that we can (normally, in an orderly manner) extrapolate a frame beyond the 
borders, it is necessary to remove these bands. 

The algorithm for the procedure for detection and removal of frame defects (dark bands) 
along the edges of the frame: 

1 . The average brightness values are calculated for m bands of a width of 1 pixel. 

height-l 

AverageY(x) = ( £Y(x, y)) I height 

y =0 - for vertical bands 

width-l 

AverageY(y) = ( £ Y(x, y)) / width 

*=° - for horizontal bands. 

2. A band is considered dark if the following conditions are fulfilled: 
if 

AverageY[i]<20 



or 



(AverageY[i+l] - AverageY[i])>35. 
3. The brightness values for pixels of a dark band are replaced by values for the pixels 
from the first non-dark band encountered, as shown in Figure 4b. 



Description of the extrapolation procedure: 
Extrapolation algorithms are used for the prediction of the brightness of pixels outside 
the borders of the frame of the video image. The given procedure is carried out after the 
procedure for cleansing frame of the dark band, if it is present in the frame. For 
extrapolation a filter of length 4 is used. 
Input data : 



Image 


Input image (frame) 


Height 


Frame height 


Width 


Frame width 


Number of points 


Number of interpolation points 




Reference points 


k 1 ,k 2 ,k 3 ,k 4 


Filter coefficients 


h 


Extrapolated point 



Output data : 



Extrapolatedjhnage 



Extrapolated image 



Extrapolation procedure algorithm: 

1 . The image is transformed into the YUV format. The algorithm is applied to each 
layer. 

2. Four reference points are necessary for the filter, according to which we will 
extrapolate new points. 

3. If we take the base points ^1*^2*^3*^4 ? then *° is point which we will 
extrapolate. 

4. Let *n*2>*3>*4 be the filter coefficients. Then the extrapolated point is 
calculated by the following method: 



5. The important feature of the given algorithm is the selection of filtration 

coefficients, the largest coefficient ^ is selected, and it is increased to the brightness 
value of the outermost pixel of the frame, as shown in Figure 5. 

Description of the procedure for selection of the most appropriate block (Best Block 

Selection) 

For selection of the most appropriate ("best") block for the given block it is important 
to have a metric that allows one to detect a compatibility of blocks and that does not 
require large computer expenditures. In the method that is the subject of this patent, 
the following metric is used: 

SAD = SAD(Y Xq h , Yx x ,y x , block _ size _ x, block _ size _ y) + 

4 • SAD(U Xq , 2 / 2 , U Xl i i,y { 1 2 , block size _x/2 9 block _ size y 1 2) + 

4 • SAD(V n f2 , V Xi /2 t y t 12 , block _ size _x/2, block _ size _yl2) 

where 

SAD(I XQ y<j Jx x ,y x , block _size _x, block _size _y) = 

^block size _y -rpblock size _x 



Z block _svze _y ^-^biock 
;=0 Z-d y=0 



where and are comparable blocks from frames I and I . The coordinates for 
the blocks are (xo,yo) and (xi,yi), respectively. The given coordinates are coordinates for 
the left upper corner of the block. 

block_size_x and block_size_y - are the measurements of the blocks. 
Y - luminance, U, V -chrominance. 



Description of the procedure for splitting the frame into quadratic blocks 

Prior to execution of the search, the next frame is covered with non-overlapping quadratic 

blocks of size NxN, where N is selected depending on the size of the frame: 



Frame format 


Frame size in pixels 
(Height) 


N 


QCIF 


144 


4 


SIF 


240 


8 


BT 


480 


16 


R601 


480 


16 



During a search of the vectors a comparison is conducted for 16x16 blocks obtained from 
NxN blocks by means of adding to them a layer of surrounding pixels, as shown in 
Figure 8. 

Depending on the frame sizes the following conditions are developed: 

N=4; 

if (height>200) 
{ 

N=8; 

} 

if (height>400) 
{ 

N-16; 

} 

Naturally, one may verify both frame height and width. Thus, depending on the format of 
the source video sequence an automatic adjustment of the system parameters, which is 
the subject of this patent, is implemented. 

Calculation of the coordinates of the lower left corner of a 16x16 block takes place 
below. 



Input data : 





Coordinates for the lower left corner of 
a NxN block 


xi,y 2 


Coordinates for the lower left corner of 
a 16x16 block 


width 


Frame width 


height 


Frame height 



The algorithm for calculation of the coordinates of the lower left corner of a 16x16 block: 

1. xi=x-(16-N)/2; 

2. yi =y-(16-N)/2; 

2. Verification is made of the following conditions: 
ifxjO =>xi=x; 
ifyi<0 => yi=y; 

ifx t +(16-N)>width-l => xi=x~(16-N); 

if y,+(16-N)>height-l => yi=y-(16-N); 
where x, y are coordinates for block NxN; 
where x i? yi are coordinates for block 16x16; 
width - frame width; 
height - frame height. 

Description of the procedure that determines the sequence of going over blocks during a 
search for displacement vectors. 

Since the algorithm for an adaptive accelerated search is based on displacement vectors 
that have already been located, it is clear that the result depends on the sequence for 
going over blocks. The following sequence for bypassing blocks is proposed: 

1 . Beginning with a central block, we make the by-pass in a spiral, as illustrated 
in Figure 9. 

2. Then we calculate the displacement vector for the remaining blocks. 
Description of the procedure for an adaptive accelerated search 

The described algorithm for the search procedure is applied for a search for displacement 
vectors for blocks and is applied separately for each block. 

The basic ideas used in the algorithm for the adaptive accelerated search procedure are: 
search zones, the use of vector-predictor from adjacent blocks and the preceding frame, 
criteria for a half-pause and the adaptive selection of thresholds. 

In the algorithm for the given procedure a aggregate of thresholds (Tl, T2, T3) that 
control the course of its operation is used: 



Threshold 


Threshold designation 


Tl 


Determines the continuation or completion of the search 


T2 


Determines if the number of subsequent examined/verified search zones is 
limited 


T3 


Determines the completion of the search according to the half-pause criterion 



The following variables are also used: 



zsize 


The parameter for the half-pause criterion, gives the maximum number of 
search zones during the scanning of which a mistake in a located vector may 
not be corrected; 


znum 


The maximum number of search zones around the selected center (0, 0); 


pznum 


The maximum number of search zones around the main vector-predictor 
(median); 


MinZone 


The current number of zones in which a vector was found with minimal error; 


Found 


An indication of the fact that all vector-predictors are equal to each other, 
different from (0,0), and correspond to the vector of the block positioned in the 
previous frame in the current position; 


Last 


An indicator of the final iteration; 


MinSAD 


The current located minimal error; 



Initial values for the thresholds TL T2 and T3 and the variables indicated above: 



Tl 


T2 


T3 


zsize 


znum 


pznum 


MinZone 


Found 


Last 


4*256 


6*256 


14*256 


3 


4 


4 


0 


false 


false 



Algorithm for the search procedure: 

Step 1. A rhomboid-shaped structure is built, containing the given number of search 
zones, as shown in Figure 10. 

Step 2. For the processed block, adjacent blocks in which displacement vectors have 
already been found are selected and stored. The selected blocks are sorted according to 
the increment of errors (SAD). Next, blocks in which there is an error twice as large as 
the smallest error are eliminated. Thus an aggregate of predictors for the search is 
created - aggregate A, containing the vectors for these blocks. 
Step 3. The threshold values are calculated. Threshold Tl is selected as the minimum 
from the error values (SAD) for the adjacent blocks, selected in step 2, and the error 
values for the block from the preceding frame in the current position of the splitting 
array. 

T2 = Tl + the dimension of the block in pixels. 
The algorithm parameters are initialized. 

Step 4. The median of the vectors for the x and y coordinates for the selected adjacent 
blocks, which is considered to be the main predictor during the search, is calculated. If 
the values of all vectors in the aggregate of predictors A: 

1) coincide and are different from (0,i) and (i,0), where i is a whole number; and 

2) coincide with the value of the median, 



then the search will be conducted only in the first zone (pznum = 1) and the "Found" 
character is specified. If only one of these conditions is fulfilled, then the search will be 
conducted only in the first two zones (pznum = 2). 

The predictor forecasts the character of the movement in the given place in the 
interpolated frame, thus, specifically due to the determination of the predictor, the system 
is adjusted to accommodate the determined character of movement in the given adjacent 
frames of the video sequence, used for the interpolation of new frames. 
Step 5, The error (SAD) for the main predictor is calculated. If the main predictor 
coincides with the vector of the block positioned in the preceding frame in the same 
position as the main predictor, but in this case the predictor error (SAD) is less than the 
error for the indicated vector, or the error according to the value is less than the 
dimensions of the block, then we move to the final step. 

Step 6. We calculate the error (SAD) for all vectors in the aggregate A and select with 
the current value the vector with the minimal error MinSAD. 
Step 7. We verify the condition MinSAD < Tl . If it is fulfilled, we go on to the final 
step. Or if the current vector coincides with the vector for the block located in the 
previous frame in the same position, but the current minimal error in this case is less, then 
we also go on to the final step. 

Step 8. If Tl < Min SAD < T2, then we in fact establish the character "Last". 
Step 9. The given number of zones is constructed around the main predictor. Then, in 
the subsequent steps each of the constructed zones is processed in sequence, beginning 
from the center. 

Step 10. We calculate the error (SAD) for all points from each of the zones. 

Step 1 1 . Criteria for the half-pause. A verification is made that the result was improved 

within the framework of the given number of the nearest already-processed zones zsize. 

If the improvements took place already a long time ago (the current zone - MinZone > 

size) and MinSAD < T3, then we go on to the final step. 

Step 12. If already in the second search zone there is no minimal error value and 

MinSAD < T3, then we go on to the final step. 

Step 13. If MinSAD < Tl or in fact is the character "Last", then we go on to the final 
step. 

Step 14. We carry out the activity of step 8. 

Step 15. We go on to the processing of the next furthest from the center zone and to step 
10. 

Step 16. Steps 9 to 15 are repeated, but this time the center zone moves to the point (0,0). 
Step 17. Radar search. Steps 9 to 14 are repeated, but this time the center zone moves to 
the point, the coordinates of which are given by the best vector found up to the current 
moment. This step is carried out the given number of times. 

Final step. The optimal vector of movement is obtained for the given block with minimal 
error MinSAD. 

Description of the procedure for calculating the values of pixels with non-integer 
coordinates. 

The given procedure is carried out in the event that the block has a displacement vector 
with non-integer coordinates, that is it is displaced by a fractional number of pixels. 
Thus, the necessity for calculation of the values of the intermediate pixels for blocks 



located in the original adjacent frames of the video sequence arises. The values of the 
intermediate pixels are calculated with the help of a bilinear interpolation formula: 
I(x+dx, y+dyH(x s y)(l-dx)(l-dy)+I(x+l, y)dx(l-dy)+ I(x, y+l)(l-dx)dy+ I(x+1, 

y+l)dxdy; 

Where I(x, y)- is the value of the pixel, 

x, y - are the coordinates of the pixel. 

The obtained result is rounded up to the nearest whole number. 

Description of the procedure for filling the interpolated frame on the basis of the 
calculated vectors of movement. 

1 . We superimpose on the interpolating frame the same array NxN as in the 
subsequent frame. 

2. For each block, we place in correspondence the displacement vector obtained 
after execution of the adaptive accelerated search procedure. 

3. We fill in the image pixels with the values 

I_interp(x, y, vx, vy, k, n) = ((n-k)I_prev(x+kvx/n, y+kvy/n)+kl_next(x-(n- 

k)vx/n, y-(n-k)vy/n))/n, where 

n is the coefficient for increasing fps; 

0<k<n is the number relative to the preceding, inserted frame; 

I_interp are the points of the interpolated frame; 

I__prev - are the points of the preceding frame; 

I_next - are the points of the subsequent frame; 

x,y - are the coordinates for the pixel in the interpolated frame; 

vx, vy - are the coordinates for the displacement vector, found upon 

fulfillment of the adaptive accelerated search procedure. 

4. For points with a fractional vector of displacement we use a bilinear 
interpolation. 

Description of the procedure for determining the presence of a scene change. 
We shall examine the initial video sequence as an aggregate of scenes. In this case we 
shall understand scene to mean a part of a video sequence in which it is possible to form 
each subsequent frame on the basis of the previous frame with the assistance of a motion 
estimation procedure, whereby the percent of blocks with an error higher than some 
threshold value must be no larger than the specified amount. If the presence of a change 
of scene is detected, then either the preceding or subsequent frame is duplicated. 



Algorithm for the procedure for determining the presence of a scene change: 



In the algorithm the following variables are used: 



cnt bad blk 


The number of vectors with an error less than some threshold 


block cnt 


The number of blocks into which the frame is split 


err[i] 


The error for the I block 


scale 


The coefficient for increasing fps 


pos 


The number for the inserted frame relative to the preceding frame, with 
which the reference is begun with zero. 



In the given invention the following algorithm is proposed for the determination of the 
scene change: 

int cnt_bad_blk=0; 

for (int i=0; i< block cnt;i++) 
{ 

if(err[i]>15*256[i]) 
{ 

cnt_bad_blk++; 

} 

} 

bool scene js changed = (cnt_bad„blk > 0.7*biock„cnt); 

If the variable scene„is„changed has the value true, then a change of scene is 
considered to have taken place, and in this case either the preceding or the subsequent 
frame is inserted as the interpolating frame. This is implemented with the help of the 
following procedure: 

if(scene_is_changed) 

{ 

if(pos<scale/2) 

{ 

We insert the preceding frame 

} 

else 
{ 

We insert the subsequent frame 



i ♦ to 

t 

Description of the procedure for deciding on the type of interpolation 
In the given procedure, the decision is adopted as to the type of interpolation for the 
given segment of the video sequence, that is either the interpolation is implemented with 
regard to motion vectors (motion compensated interpolation), or a pixel by pixel 
interpolation is implemented. 

Pixel-by-pixel interpolation is carried out with the help of the following formula: 

I_interp(x, y, k, n)=((n-k)I_prev(x, y)+kl_next(x, y))/n, 
where IJnterp is the value of the interpolating pixel; 
I_prev - is the value of the pixel in the preceding frame; 
Inext - is the value of the pixel in the next frame; 
x, y - are the pixel coordinates; 
n - is the coefficient for increasing fps; 

k - is the number relating to the preceding, inserted frame, 0<k<n. 



The following variables are used in the algorithm: 



cnt 


The number of vectors with an error less than some 
threshold 


average_v_x, 
average v y 


Coordinates of the average vector 


block cnt 


The number of blocks into which a frame is split 


vx[i], vy[i] 


Coordinates of the displacement vector for block i. 




The error for block i 


maxjj 


The maximum value for the x -components of a 
vector with an error greater than some threshold 


max_v_y 


The maximum value for the y-components of a 
vector with an error greater than some threshold 


maxv 


The larger of the two values max v x and max v_y 



1 . The average vector is calculated with the help of the following algorithm: 



cnt=0 
average_v_x=0 
average_v_y=0 

for (int i=0; i<block_cnt; i++) 
{ 

if(err[i]<15* 256[i]) 



a v erage_ v_x+ = v x f i] ; 
a verage_ v_y + = v y [ i] ; 
cnt++; 

} 

} 

averages .x / =cnt; 
avera ge_ v . y / =cnt; 



2, The decision about the type of interpolation is made with the help of the following 
procedure: 

max_v_y=0; 

for (int 1=0; i<block„cnt;i++) 
{ 

if(err[i]<15*256[i]) 
{ 

cnt++; 

} 

if(err[i]>15* 256[i]) 
f 

if(abs(vv[i])>max_v_v) 

{ 

max_v_v=abs(w [i]); 

} 

if(abs(vx [i])>max_y_x) 

{ 

max„v_x=abs(vx [i]); 

} 

} 

} 

nniax_v_x=abs(max_v_x-abs(average_v_x)); 
max._v_y=abs(max3\y-abs(average„v_y)); 
maxv=max(max__v_x, max_v__y) 
not_use_MCI = 
(maxv>(SegmJmg>height/32)& 



If the value of the variable not_useJV!CI is true, then we use the pixel-by-pixel 
interpolation described above; in the opposite case we use interpolation with regard to 
vectors of movement (motion compensated interpolation). 

In this invention a method and device for the temporal interpolation of video sequences 
are described. 

In accord with patent legislation, the invention and its structural and methodological 
characteristics are described with specific language. 

This invention is not limited by the description set forth above, as various modifications 
and variations of its essence and the contents of the claims for this invention are possible. 



